Periodic Remapping of Non-Deterministic Bulk Synchronous Computations

Dynamic remapping is critical to the performance of bulk synchronous computations that have non-deterministic behaviors because of the need of barrier synchronization between phases. This paper presents a formal analysis of the issue of when to remap for dynamic computations with a priori known statistical behaviors, with an objective of finding optimal remapping frequencies for a given tolerance of load imbalance. The problem is formulated as a sequential stochastic optimization. Since general optimization techniques tend to reveal stationary properties of the workload process, they are not readily applicable to the analysis of the effect of periodic remapping. Instead, this paper develops new analytical approaches to precisely characterize the transient statistical behaviors of the workload process on both homogeneous and heterogeneous machines. Nearoptimal remapping frequencies are derived for various random workload change processes with known or unknown probabilistic distributions. The approximations are shown accurate via simulations.